• Unsupervised Machine Learning
• Dimensionality Reduction Technique
• Data Exploration
• Feature Extraction
• Data Visualization

• Simplification of complex dataset
• Principal Component (PC): Capture variances explaining the original variables
• Mitigate multicollinearity

• Data matrix \(X\) of size \(N\) x \(P\).
• Data is linearly related.
• Continuous and normally distributed data.
  • Initial data distribution does not truly matter.
• Variables are similar in scale and without extreme outliers.

• Missing data: Imputation or removal of observations.
• Centering and scaling: Transform variables to a mean of 0 and a standard deviation of 1. \[ z_{np} = \frac{x_{np} - \bar{x}_{p}}{{\sigma_{p}}} \]

• Covariance: A measure of how two random variables vary together. \[ Cov(x,y) = \frac{\Sigma(x_i-\bar{x})(y_i-\bar{y})}{N} \]
• Covariance Matrix: Symmetric \(p \times p\) matrix which gives the covariance values for each pair of variables in the dataset.

• Nonzero vector whose direction is unaffected by a linear transformation.
• An eigenvector is scaled by factor \(\lambda\), the eigenvalue.
• Each principal component is given by the eigenvectors of the covariance matrix.
  • The eigenvectors represent the directions of the new principal axes.
  • The eigenvalues represent the magnitude of these eigenvectors.

• Find the linear combination of the columns of \(X\) (the variables) which maximizes variance.
• Let \(a\) be a vector of constants \(a_1, a_2, a_3, …, a_p\) such that \(Xa\) represents the linear combination which maximizes variance.

• The variance of \(Xa\) is represented by \(var(Xa) = a^TSa\) with the covariance matrix \(S\).
• Finding the \(Xa\) with maximum variance equates to finding the vector \(a\) which maximizes the quadratic \(a^TSa\), where \(a^Ta = 1\).

• \(a\) is a unit-norm eigenvector with eigenvalue \(\lambda\) of the covariance matrix \(S\).
• The largest eigenvalue of \(S\) is \(\lambda_1\) with the eigenvector \(a_1\), which we can define for any eigenvector \(a\): \[ var(Xa) = a^TSa = \lambda a^Ta = \lambda \]

• Impose the restriction of orthogonality to the coefficient vectors of \(S\).
  • Ensure the principal components are uncorrelated.
• The eigenvectors of \(S\) represent the solutions to finding \(Xa_k\) which maximize variance while minimizing correlation with prior linear combinations.
• Each \(Xa_k\) is a principal components of the dataset having eigenvectors \(a_k\) and eigenvalues \(\lambda_k\).

• The elements of \(Xa_k\) are the factor scores of the PCs.
• The elements of the eigenvectors \(a_k\) represent the loadings of the PCs.

• The elements of \(Xa_k\).
• How each observation scores on a PC.
• In a geometric interpretation of PCA the factor scores measure length (magnitude) on the Cartesian plane.
• This length represents the projection of the original observations onto the PCs from the origin at \((0, 0)\).

• The elements of the eigenvectors \(a_k\) represent the loadings of the PCs.
• The loadings represent the weights of the original variables in the computation of the PCs.
• The correlation from -1 to 1 of each variable with the factor score.

• Eigenvectors: Represent directions of maximum variance.
• Eigenvalues: Indicate the variance explained by each eigenvector.
• Sorting: Sort eigenvalues in descending order to select the most significant principal components.

• Efficient removal of white spaces in the dataset.

• Editing variable names to enhance readability and meaningful.

Original: “Percentage.Of.Adult..Patients.With.Hypercalcemia..Serum.Calcium.Greater.Than.10.2.Mg.dL.”

Edited: “hypercalcemia_calcium > 10.2Mg.”

• Principal component analysis was performed using a singular value decomposition approach.
• PC1 captures 40.80% of the variance in the data.
  • PC1 and PC2 capture 50.27% of the variance.
    • The first four PCs capture 67.66% of the variance, or just over two-thirds.
• After the fourth PC, the variance captured by each successive PC begins to diminish relative to PCs one through four.
  • The first ten PCs capture 88.67% of the variance.
    • Over 90% of the information in the dataset can be explained by the first eleven PCs.

• The variables which contribute the most to PC1 are
  • expected_hospital_readmission
  • expected_transfusion
  • expected_hospitalization
• PC2, which is orthogonal to PC1, has relatively large contributions from the five variables measuring levels of phosphorus.

• Principal component regression was performed with expected_survival used as the response variable.
• The estimates and significance of each PC regressor demonstrates the differences between variance captured from the data and usefulness in a linear model.
  • For example, PC4 is a significant regressor despite capturing less variance than PC3 in the training data.
• Both models produced an \(R^2\) above 96% and a predicted \(R^2\) above 95% with a 1% advantage on the cross-validation model.